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ABSTRACT 

A job analysis was conducted to define the knowledge 
domain in which newly licensed (certified) mathematics teachers must 
be knowledgea le to perform competently. The results of the job 
analysis were to be used to develop test specifications for the 
Subject Assessment in Mathematics of the Praxis Series: Professional 
Assessments for Beginning Teachers. A draft knowledge domain of 12 
knowledge categories and 175 statements was constructed at the 
Educational Testing Service and submitted to review by 11 
subject-matter experts. The revised draft domain was reviewed by an 
Advisory Committee of secondary school teachers, college faculty, and 
an admini s t rat or , and their revisions were reviewed through a 
national survey of 500 teachers, 250 college faculty, and 50 school 
admi ni s t rators , followed by review by an additional 200 new teachers. 
A cut point was established to identify the core of important 
statements. A total of 65 statements of the 193 submitted to the 
national sample did not meet the criterion for inclusion. The 
remaining statements, finally grouped into 13 knowledge areas, were 
to be used as the basis for test specif ications. Seven appendixes, 
with three appendix tables, provide supplemental information about 
the study, including the survey questionnaire. (Contains 5 tables and 
14 references.) (SLD) 
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Executive Summary 



A job analysis was conducted to define the knowledge domain in which newly licensed 
(certified) mathematics teachers must be knowledgeable to perform their jobs in a competent 
manner. The re.sults of the job analysis will be used to develop test specifications for the Subject 
Assessment in mathematics of the Praxis Series: Professionals Assessments for Beginning 
Teachers™. 

A draft knowledge domain was constructed by Educational Testing Service (ETS) Test 
Development staff who have subject-matter expertise in mathematics and ETS Research staff 
who have expertise in job analysis. In the pn ;ess of developing the draft domain, ETS subject- 
matter experts reviewed state licensure (certification) requirements, previous National Teacher 
Examination (NTE) mathematics test specifications, current test items, relevant professional 
literature, and standards from the National Council for Teachers of Mathematics (NCTM). The 
resultant draft domain consisted of 12 major knowledge categories and 175 statements. The 12 
major categories were: (1) Basic Mathematics, (2) Geometry, (3) Trigonometry, (4) Functions 
and Their Graphs, (5) Probability and Statistics, (6) Calculus, (7) Analytic Geometry, (8) 
Discrete Mathematics, (9) Abstract Algebra, (10) Linear Algebra, (11) Computer Science, and 
(12) Pedagogy Specific to Mathematics. The 175 statements were operationalized in behavioral 
terms and, hence, referred to as tasks. 

This draft domain was then reviewed by an External Review Panel of 11 subject-matter 
experts: five classroom teachers, three college faculty, two school administrators, and one 
person from the NCTM. The panel reviewed the draft domain for (1) the appropriateness of its 
overall structure and (2) the appropriateness of the specific task statements and their 
completeness and clarity. Revisions suggested by the panel, including additions and deletions of 
knowledge categories and statements, were obtained via telephone interviews conducted by ETS 
Research staff. Based on the interviews, wording changes were made to 66 of the 175 
statements, and nine new statements were added to the draft domain. 

This revised draft domain was then reviewed by an Advisory Committee. The committee 
was comprised of two secondary school mathematics teachers, four college faculty, and one state 
administrator. The committee was charged with modifying the revised draft domain so that it 
accurately reflected what the members believed were important knowledge areas for newly 
licensed (certified) mathematics teachers. This modification process occurred during a four-day 
meeting held at ETS. During their review, the committee suggested considerable modification 
of the inventory. As a result, changes to 87 of the 183 statements in the draft were made. In 
addition, 16 statements were deleted and 25 statements were added. Among the additions were 
11 statements comprising a new category called Mathematical Reasoning and Modeling. The 
committee also changed the name of the Basic Mathematics category to Arithmetic and Basic 
Algebra. The resulting domain consisted of 193 statements. 

This revised domain was then subjected to verification/refutation through a national survey 
of 500 teachers (10 per state), 250 college faculty (5 per state), and 50 school administrators (1 
per state) for a total of 800 education professionals (16 per state). The mailing list was made 
up of names from the NCTM membership roster so that appropriate people could be reached. 
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Names from the roster were drawn at random in a way that satisfied the state participation 
requirements stated above. 

Teach' rs were included in the original sample, and we later sent the survey to an additional 
200 teachers. In this supplemental sample we attempted to focus on individuals who were 
relatively new to the teaching profession (e.g., less than five years teaching experience). We did 
this to increase the likelihood that a sufficient number of responses from new teachers would be 
available for analysis. 

All survey participants were asked to rate the statements in terms of their importance for 
newly licensed (certified) mathematics teachers to perform their jobs in a competent manner. 
Respondents used a 5-point scale ranging from a low of 0 (of no importance) to a high of 4 
(very important) to make their judgments. The purpose of the .‘iurvey administration was to 
identify a core of knowledge statements that relatively large numbers of education professionals 
verified to be important for newly licensed (certified) mathematics teachers. The latter 
objective is accomplished through the analysis of the mean importance ratings provided by three 
groups of education professionals (i.e., teachers and college faculty in the original sample and 
teachers in the supplemental sample) and by appropriate subgroups of respondents (sex, 
race/ethnicity, geographic region, teaching experience in the two samples combined). Task 
statements that are judged to be important by all respondent groups and subgroups define the 
core. The core becomes the primary data base for the development of test specifications. The 
derivation of test specifications from those statements verified to be important by those 
sur\'eyed provides a substantial evidential basis for the content validity (content relevance) of 
The Praxis II Subject Assessment in mathematics. 

Two types of data analysis were conducted to support the des'elopment of content valid 
(content relevant) test specifications for the Subject Assessment in mathematics: (1) means 
were computed of the importance ratings for each task statement by the three groups of 
education professionals and by the appropriate subgroups of respondents; and (2) correlations of 
the profiles of these mean importance ratings were computed across the three groups of 
education professionals and within the appropriate subgroups of respondents. 

A cut point of a mean importance rating of 2.50 (the midpoint between moderately 
important [scale value 2] and important [scale value 3]) was established to identify the core of 
important statements. Statements that were judged by the three groups of education 
professionals and all subgroups of respondents to be 2.50 or higher comprised the core and 
therefore were considered eligible for inclusion in the development of test specifications. 
(However, because the survey participants were not involved in the development of the 
knowledge domain, they may lack certain insights that the Advisory Committee members have 
due to their high level of involvement in the domain’s development. As a consequence, if the 
committee believes that a task statement rated below 2.50 should be included in the 
specifications and the committee can provide compelling written rationales, those task 
Statements may be reinstated for inclusion in the test specifications.) 

The results of the mean analysis conducted by teachers (primary and supplemental samples) 
and college faculty (primary sample) showed that 64 knowledge and ability statements were 
rated less than 2.50. This represents .33.2% of the content domain. In the subgroup analyses, 63 
(32.6%) statements were rated below 2.50. In total, 65 of the 193 statements (33.7%) did not 
meet the 2.50 criterion for inclusion. Most of the statem.ents were in the knowledge categories 
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repieseiiting upper-level mathematics (i.e., Calculus, Probability and Statistics, Discrete 
Mathematics, Abstract Algebra, and Linear Algebra). 

The computation of correlation coefficients to assess agreement in terms of perceived 
relative importance of the task statements revealed a very high level of agreement. The 
coefficients for comparisons among the two sets of teachers and college faculty exceeded .90, as 
did the coefficients generated during the subgroup analyses. These findings indicate that there is 
substantial agreen >t on the relative importance given to the tasks by a diverse group of education 
professionals. 

The 128 task statements that were verified to be important by the surveyed teachers, the college 
faculty, and the demographic subgroups should be used as the foundation for the development of test 
specifications. Test specifications that are linked to the results of a job analysis provide support 
for the content validity of the derived assessment measures and may be seen as part of an initial 
step in ensuring the fairness to subgroups of mathematics teacher candidates of the derived 
assessment measures. It is reasonable to assume that, due to testing and psychometric 
constraints (e.g., time limits, ability to measure content reliably), not all of the verified content 
will be included on the assessment measures. One source of information that may be used to 
guide the Advisory Committee in their decision as to what verified content to include on the 
assessment measures is the mean importance rating. Although a rank ordering of the content 
b}' mean importance rating is not implied, it is recommended that initial consideration be given 
to content that is well above the cut point and represents the appropriate breadth of content 
coverage. 

Evidence was also provided in this study of the comprehensiveness of the content domain 
within the 13 major knowledge categories. This information serves as a check on the adequacy 
of the content domain definition. If the domain was well defined, then the knowledge categories 
should be judged to have beon well covered by their accompanying task statements. The results 
supported the adequacy of ti'e content domain definition. 
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Introduction 



Purpose of the Study 

The subject assessments for The Praxis Series: Professional Assessments for Beginning 
Teachers™ are designed to assess a prospective teacher’s content knowledge of a specific subject 
area and subject-specific pedagogical knowledge. The focus of such tests is based on the 
premi; e that beginning teachers should demonstrate knowledge of the subjects they intend to 
teach (Grossman, Wilson, & Shulman, 1989) and, perhaps, demonstrate knowledge of teaching 
principles, strategies, and resources specific to those subjects (Grossman, 1989; McDiarmid, Ball, 
& Anderson, 1989; Reynolds, 1992). The Praxis Series can be used by state agencies as one of 
several criteria for initial teacher licensure (certification). Included as part of the subject 
assessments is a licensure examination for mathematics teachers. To identify the content 
domain of this examination and to support the content validity (content relevance) of this 
examination, a job analysis was conducted of the knowledge important for newly licensed 
(beginning) mathematics teachers. This report will describe the job analysis study. In particular, 
it will provide the rationale for conducting the job analysis, present the methods used to define 
job-related knowledge, describe the types of statistical analysis conducted, report the results of 
these analyses, and specify the implications of the results for developing test specifications. 



Standards for Educational and Psychological Testing 

The Standards for Educational and Psychological Testing (1985) is a comprehensive technical 
guide that provides criteria for the evaluation of tests, testing practices, and the effects of test 
use. It was developed jointly by the American Psychological Association (APA), the American 
Educational Research Association (AERA), and the National Council on Measurement in 
Education (NCME). The guidelines presented in the Standards have, by professional consensus, 
come to define the necessary components of quality testing. As a consequence, a testing 
program that adheres to the Standards is more likely to be judged to be valid (defensible) than 
one that does not. 

There are two categories of cr iteria within the Standards, primary and secondary. Those 
classified as primary "should be met by all tests . . . unless a sound professional reason is 
available to show why it is not necessary, or technically feasible, to do so in a particular case. 
Test developers and users ... are expected to be able to explain why any primary standards 
have not been met" (AERA, APA, & NCME, 1985, p. 2). One of the primary standards is that 
the content domain of a licensure or certification test should be defined in terms of the 
importance of the content for competent performance in an occupation. "Job analyses provide 
the primary basis for defining the content domain." (p. 64) 

The use of job analysis to define the content domain is a critical component in establishing 
the content validity of licensure and certification examinations. Content validity is the principal 
validation strategy used for these examinations. It refers to the extent to which the content 
covered by an examination overlaps with the important components (tasks, knowledge, skills, or 
abilities) of a job (Arvey & Faley, 1988). Demonstration of content validity is accomplivlicd 
through the judgments of subject-matter experts. It is enhanced by the inclusion of large 
numbers of subject-matter experts who represent the relevant areas of expertise (Ghiselli, 
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Campbell, & Zedeck, 1981). The lack of a well-designed job analysis is frequently cited by the 
courts as a major cause of test invalidity. 



Job Analysis 

Job analysis refers to procedures designed to obtain descriptive information about the tasks 
performed on a job and/or the knowledge, skills, and abilities thought necessary to adequately 
perform those tasks (Gael, 1983). The specific type of job information collected for a job 
analysis is determined by the purpose for which the information will be used. For purposes of 
developing licensure and certification examinations, a job analysis should identify the important 
knowledge or abilities necessary to protect the pu6/zc— interpreted as the importance of the content 
for competent performance in an occupation (AERA, APA, & NCME, 1985). In addition, a 
well-designed job analysis should include the participation of various subject-matter experts 
(Mehrens, 1987); and the data collected should be representative of the diversity within the job. 
Diversity refers to regional or job context factors and to subject-matter-expert factors such as 
race/ethnicity, experience, and sex (Kuehn, Stallings, & Holland, 1990). The job analysis 
conducted for mathematics teachers was designed to follow guidelines presented in the 
Standards for Educational and Psychological Testing and to adhere to accepted professional 
practice. 



Objectives of the Job Analysis Study 

The objectives of this study were: (1) to construct a comprehensive domain of knowledge 
that is in portant for newly licensed (certified) mathematics teachers and then (2) to obtain, 
using survey methodology, the independent judgments of a national sample of mathematics 
education professionals (teachers, college faculty, and state administrators) to verify or refute 
the importance of the domain of knowledge. The verification/refutation component plays a 
critical part in ensuring that the domain (in whole or in part) is judged to be relevant to the job 
of a newly licensed (certified) mathematics teacher by a wide array of education professionals. 
The components of the domain that are verified will be used in the development of test 
specifications for The Praxis II Subject Assessment in mathematics. 



Methods 

The job analysis study described in this report involved a multi-method approach that 
included subject-matter experts and a national survey. First, groups of subject-matter experts 
defined a knowledge domain important for newly licensed/certified mathematics teachers. A 
description of this knowledge domain was then sent out to education professionals through a 
large-scale national survey. The purpose of the survey administration was to: (1) obtain 
verification and/or refutation from large numbers of education professionals that the previous 
groups of subject-matter experts had defined a domain of knowledge important for newly 
licensed mathematics teachers. Through this pt ocess we can identify a core of important 
knowledge that is related to the job of the newly licensed mathematics teacher. The survey 
functions as a "check and hahince" on the judgments of the subject-matter experts and reduces 
the likelihood that unimportant knowledge areas will be included in the development of the test 
specifications. The use of a job analysis survey is also an efficient and cost-effective method of 
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obtaining input from large numbers of subject-matter experts and makes it possible for ratings to 
be analyzed separately by appropriate subgroups. 

The survey participants were mathematics teachers, administrators, and college faculty 
whose names were obtained from the membership of the National Council of Teachers of 
Mathematics (NCTM). An additional list of mathematics teachers was obtained from Market 
Data Retrieval, an education mailing list company. The participants were asked to rate the job 
analysis task statements in terms of their importance for newly licensed/certified mathematics 
teachers to perform their jobs in a competent manner. The specific steps in the job analysis 
process are described below. 



Definition of the Knowledge Domain 

Development of a draft knowledge domain . The first step in the process of conducting the 
job analysis was to construct a preliminary knowledge domain. The domain was constructed by 
Educational Testing Service (ETS) Test Development staff who have subject-matter expertise in 
mathematics and ETS Research staff who have expertise in job analysis methodology. In the 
process of developing the draft, the ETS subject-matter experts reviev/ed state licensure 
(certification) requirements, previous National Teacher Examination (NTE) mathematics test 
specifications, current test items, relevant professional literature, and standards from NCTM 
(i.e.. Curriculum and Evaluation Standards for School Mathematics, 19b9). 

The resultant draft domain consisted of 12 major knowledge categories: (1) Basic 
Mathematics, (2) Geometry, (3) Trigonometry, (4) Functions and their Graphs, (5) Probability 
and Statistics, (6) Calculus, (7) Analytic Geometry, (8) Discrete Mathematics, (9) Abstract 
Algebra, (10) Linear Algebra, (11) Computer Science, and (12) Pedagogy Specific to 
Mathematics. Within each category were several statements mapping the important aspects of 
the category. These statements were presented in behavioral terms (e.g., order any finite set of 
real numbers and recognize equivalent forms of a number [e.g,, 1 [10, 0.1, 10 '}', discuss informally 
what it means for a function to have a limit at a point). Because the statements were presented 
in behavioral terms, they were referred to as tasks. This draft, included a total of 175 task 
statements for the 12 categories. 

Evaluation of draft domain by External Review Panel . Consistent with a content validity 
framework, the job analysis study was designed to obtain input from many subject-matter experts 
at several critical points in the domain definition process. To this end, an External Review 
Panel of 1 1 mathematics education professionals was formed to review the draft domain. This 
group consisted of five classroom teachers, three college faculty, two school administrators, and 
one individual from the NCTM. Individuals were considered for membership through a process 
of peer recommendation. AH of the review panelists have experience either teaching 
mathematics or supervising teachers of mathematics. Generally, they are prominent and active 
in professional associations and/or teacher licensure. In addition to their subject-matter 
expertise, the panel was formed so as to have representation by sex, ethnicity, and geographic 
location. Members of the panel are listed in Appendix A. 

The panelists were instructed to review the draft and to make the modifications they felt 
were necessary to adequately cover the important aspects of teaching mathematics. They were 
further instructed that these modifications could include the addition of important task 
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statements, deletion of unimportant statements, elaboration of statements with relevant 
examples, and revision of statements so that the language would be clear and appropriate for 
individuals in mathematics education. The panelists were interviewed via telephone by ETS 
Research staff to obtain their suggestions for modifications. 

Information from the interviews was compiled, discussed with ETS Test Development Staff, 
and, subsequently, used to revise the draft. Of the 175 statements in the first draft, 66 were 
modified based on these interviews. Also, nine new statements were incorporated. 

Advisory Committee meeting . The next step in the job analysis process was a meeting held 
September 22-25, 1989, in Princeton, New Jersey, with an Advisory Committee of seven subject- 
matter specialists. The committee was charged with developing a final version of the Task 
Analysis Inventory and with developing the specifications for the new test. Like the external 
review panelists, members of the Advisory Committee have documented knowledge of the 
subject matter. The committee comprises two classroom teachers, four college faculty members, 
and one school administrator and has representation by sex, ethnicity, and geographic locijtion. 
Members of the committee are listed in Appendix B. 

The meeting was led jointly by ETS Test Development and Research staff. Prior to the 
meeting, committee members were maOed a copy of the draft domain to review. They were 
informed about the purpose of the meeting and asked to come prepared to discuss their review. 
Because they will use the results obtained from a survey administration of the content domain, it 
is critical that committee members have a clear understanding of each statement. The group 
interaction during the meeting fostered discussions that generated suggestions not made during 
the individual interviews with the External Review Panelists. The committee members 
attempted to be inclusive (i.e., cover all important aspects of teaching the subject matter) rather 
than exclusive in defining the content domain. 

During their review, the committee suggested considerable modification of the inventory. 

As a result, changes to 87 of the 183 statements in the draft were made. In addition, 16 
statements were deleted and 25 statements were added. Among the additions were 11 
statements comprising a new category called Mathematical Reasoning and Modeling. 

During the meeting, the Advisory Committee also reviewed and approved the proposed 
rating scale for the inventory. The rating scale required respondents to make judgments 
reg irding importance to the newly licensed teacher. The importance scale, which is shown 
below, is in compliance with professional standards (cf. AERA, APA, & NCME, 1985). 



How important is it that a newly licensed fcertifiedJ mathematics teacher be 
able to perform this task in a competent manner? 

(0) Of no importance 

(1) Of little importance 

(2) Moderately important 

(3) Important 

(4) Very important 
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The committee also reviewed and approved items concerning demographic and background 
information (e.g., sex, teaching experience, geographic location). Such items were included so 
that we could describe the composition of the survey respondent group and conduct analyses of 
the survey responses by various subgroups of respondents (e.g., males and females). 

Pilot test of the Task Analysis Inventory . After the meeting, a revised Task Analysis 
Inventory was mailed to the committee members for final approval. Once approval was 
obtained, the inventory was pilot tested on a group of nine classroom teachers. The pilot 
participants were asked to review the survey for clarity of wording, ease of use, and 
comprehensiveness of content coverage. The pilot test indicated that no one had difficulty 
completing the inventory and that no additional changes were necessary. 



Large-Scale Survey 

Survey instrument . The finalized survey consisted of three parts. Part I included 13 major 
Knowledge categories: Arithmetic and Basic Algebra, Geometry, Trigonometry, Functions and 
Their Graphs, Probability and Statistics, Analytic Geometry, Calculus, Discrete Mathematics, 
Abstract Algebra, Linear Algebra, Computer Science, Mathematical Reasoning and Modeling, 
and Content-Specific Pedagogy. Under these categories were 193 specific task statements. 
Survey respondents were asked to rate the statements using the importance scale shown above. 

For each major knowledge category, there was also a content coverage question in Part I. 
Survey participants were asked to indicate how well each major knowledge ca.cegory was covered 
by its task statements. Respondents made their judgments using a 5-point rating scale 
(1 = PoorIy, 2 = Somewhat, 3= Adequately. 4 = Well, 5 = Very well). The participants also had an 
opportunity to identify and write in task statements that they believed should be added to the 
domain. 

In Part II of the survey, participants were asked to indicate the weight (emphasis) that each 
of the major knowledge categories should receive on the assessment. This was accomplished by 
distributing 100 total points across the major areas. These point distributions were converted 
into percentages, representing the percent of items that the survey respondents believed should 
be devoted to each area. 

In Part HI, participants were asked for demographic and background information. As 
previously noted, these items are used to describe the respondents and to perform subgroup 
analyses. A copy of the final survey L provided in Appendix C. 

Survey participants . The primary sample for this study consisted of 500 teachers (10 per 
state), 250 college faculty (5 per state), and 50 school administrators (1 per state) for a total of 
800 (16 per state). The mailing list was made up of names from the NCTM membership roster 
so that appropriate people could be reached. Names from the roster were drawn at random in 
such a way as to satisfy the state participation requirements stated above. 

Teachers were included in the primary sample, and we also sent the surv'ey to an extra 200 
teachers. In this supplemental sample, we attempted to focus on individuals who were relatively 
new to the teaching profession (e.g., less than five years’ teaching experience). We did this to 
increase the likelihood that a sufficient number of responses from new teachers would be 



available for analysis. It is important to survey new teachers because they are most like the 
population that will eventually take the assessment. Names were obtained from Market Data 
Retrieval (MDR), an education mailing list company. For our study, MDR was unable to 
specifically identify new teachers, but, as a surrogate strategy, was able to identify teachers who 
were new to their schools. 

Survey administration . The surveys were administered to the p.rimary sample in December 
1989. Surveys were administered to the supplemental sample in May 1991. Each survey was 
accompanied by a letter of invitation to participate and a postage-paid envelope for return of 
the completed survey. A reminder postcard was mailed approximately one week after the 
survey mailing. The cover letters for the primary and supplemental samples and the follow-up 
postcard are provided in Appendix D. 

The purpose of the survey administration was to identify a core of task statements that 
relatively large numbers of education professionals judged to be relevant (verified as important) 
to newly licensed (certified) mathematics teachers. The latter objective is accomplished through 
an analysis of the mean importance ratings provided by the three groups of education 
professionals and by the relevant subgroups of respondents. Task statements that are judged to 
be important by all respondent groups and subgroups define the core. The core becomes the 
primary data base for the development of test specifications. The derivation of test 
specifications from those knowledge and ability statements verified as important by the surveyed 
education professionals provides a substantial evidential basis for the content validity (content 
relevance) of the Subject Assessment in mathematics. 



Results 



Survey Re.spondents: Primary Sample 

Response rate . Of the 800 inventories mailed to the primary sample, 16 were returned 
incomplete for a variety of reasons (e.g., wrong address, individual was retired and declined to 
participate). Of the remaining 784, 462 (58.9%) were completed and returned. 

Demographic characteristics . Results of the analysis of the responses to the demographic 
questions are summarized in Appendix E. The typical respondent was over 35 years old. White, 
had at least a master’s degree, and had more than 20 years’ experience in teaching mathematics. 
More of the respondents were male than were female (63.0% to 34.6%). The respondents who 
taught tended to do so in high school or in college. 



Surv'ev Respondents: Supplemental Sample 

Response rate . Of the 200 inventories mailed to the supplemental sample, 1 1 were 
returned incomplete. Of the remaining 189, 82 (43.4%) were completed and returned. 

Demographic characteristics . Demographic distributions for this sample are also provided 
in Appendix E. As intended, most of the respondents were teachers (76/82 = 92.7%). Of the 
respondents, 43.9% (36/82) had 5 or fewer years of teaching experience. Hence, the sampling 
strategy was partially successful in identifying relatively new teachers. In addition to teaching 



experience, this sample was different from the primary sample on other demographic variables. 
For example, the supplemental sample tended to be younger, had a higher percentage of women 
and minority respondents, and had fewer respondents with a master’s degree than the primary 
sample. 



Analysis of Importance Ratings 

Two types of data analysis were conducted to support the development of content valid 
(content relevant) test specifications for the Subject Assessment in mathematics: (1) Means of 
the importance ratings were computed for each task statement by the three groups of education 
professionals and by the appropriate subgroups of respondents, and (2) correlations of the 
profiles of these mean importance ratings were computed across the three groups of education 
professionals and the appropriate subgroups of respondents. 

Means . The mean analysis is used to determine the level (absolute value) of importance 
attributed to the task statements. Means were computed for teachers and college faculty in the 
primary sample and teachers in the supplemental sample. Means were also computed for 
appropriate subgroups of respondents in the two samples combined (sex, race/ethnicity, 
geographic region, teaching experience). An analysis of importance ratings by geographic region 
is consistent with the recent legal emphasis on addressing regional job variability when 
conducting job analyses for content domain specification purposes (Kuehn et al., 1990). We 
used the regional categorizations established by the National Association of State Directors of 
Teacher Education and Certification (NASDTEC) in our analysis. Sex and race/ethnicity 
subgroups were included because they represent protected "classes" under Title VII of the Civil 
Rights Act of 1964. (Responses from all racial and/or ethnic minority participants were 
aggregated because of the insufficient number of respondents, i.e., < 30, from any single 
racial/ethnic group.) We used a dichotomous breakdown of teaching experience at the 5-year 
point was chosen so that the judgments of less experienced teachers and more experienced 
teachers could be represented separately. 

A respondent category' was required to have at least 30 respondents to be included in the 
mean analysis (e.g., > 30 college faculty, > 30 females). This is a necessary condition to ensure 
that the mean value based upon the sample of respondents is an accurate estimate of the 
corresponding population mean value (Walpole, 1974). Consequently, there were insufficient 
numbers of state administrators to analyze their responses separately. Task statements that 
meet or exceed a mean importance value of 2.50 (to be discussed in a later section) by all three 
groups of education professionals (teachers and college faculty in the primary sample and teachers 
in the supplemental sample) and by all subgroups of respondents may be included in the 
development of the test specifications. In addition, mean ratings were computed for the 
responses to the content coverage section and the recommendation for test content section of 
the survey. These analyses were computed for the three groups of education professionals and 
for the total sample. 

Correlations . The correlational analysis is u.sed in this study to determine the extent of 
agreement among the three groups of education professionals and among the demographic 
subgroups of respondents about the relative importance of the task statements. Relative 
importance refers to the similarity of the pattern of mean ratings generated by the different 
respondent groups. For example, the profile of 193 mean ratings for teachers in the primary 
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sample is correlated with the profile of 193 mean ratings for teachers in the supplemental 
sample. If these two profiles are similar (the shapes of the profiles are complementary), the 
value of the correlation coefficient will be close to 1.00. 



Criterion for Interpretation of Mean Importance Ratings 

As the purpose of a job analysis is to ensure that only the most important task statements 
are included in the development of test specifications, a criterion (cut point) for inclusion needs 
to be established. A criterion that was used in a similar study (i.e., Rosenfeld & Tannenbaum, 
1991) is a mean importance rating that represents the midpoint between moderately important 
and the next higher scale value. For the importance rating scale used in the present job 
analysis, the value of this criterion is 2.50 (midpoint between moderately important and 
important). It is believed that this criterion is consistent with the intent of content validity, 
which is to measure only important knowledge in the assessment measure. Therefore, task 
statements that receive a mean importance rating of 2.50 or more may be considered eligible for 
inclusion in the development of test specifications; task statements that receive a mean rating of 
less than 2.50 may not be considered for inclusion. (Because survey participants were not 
involved in the development of the content domain, however, they may lack certain insights that 
the Advisory Committee members have because of their high level of involvement in the 
domain’s development. Consequently, if the committee believes that a task statement rated 
below 2.50 should be included in the specifications and the committee can provide compelling 
written rationales, those task statements may be reinstated for inclusion in the test 
specifications.) 



Mean Importance Ratings 

Education professionals . Means and standard deviations were computed for teachers and 
college faculty in the primary sample and for teachers in the supplemental sample. Because of 
their length, these results are provided in Appendbc F. 

Those task statements rated less than 2.50 by either the teachers or college faculty in the 
primary sample or by the teachers in the supplemental sample are provided in Table 1. Of the 
193 individual task statements, 64 (33.1%) were rated below 2.50 by one or more of the three 
groups. The following dimensions yielded the highest percentages of statements with low 
ratings: Probability and Statistics (10 of 19 statements). Discrete Mathematics (17 of 21), 
Abstract Algebra (7 of 7), and Linear Algebra (10 of 12). 

Teachers gave more of the lower ratings. Both teacher samples gave average ratings below 
2.50 to 57 task statements, 48 of which were common to Loth. In contrast, the average college 
faculty rating was less than 2.50 on only 28 statements. Of the 28, all were given average ratings 
below 2.50 by the primary sample of teachers, while 27 were rated below the criterion by the 
supplemental sample of teachers. 

Demographic subgroups . Mean ratings were computed for demographic subgroups based 
on sex, race/ethnicity, geographic region, and teaching experience (for teachers only). These 
data are presented in a table in Appendix G. 



Table 1 

Statements Rated Below 2.50 by Teachers and College Faculty 







Primary 

Sample 


Supplemental 

Sample 






Teachers 
(N = 180) 


College 
Faculty 
(N = 159) 


Teachers 
(N = 77) 


ARITHMETIC AND BASIC ALGEBRA 








9 


Rnd positive integral powers and roots of perfect powers 


2.36 


2.28 




GEOMETRY 








44 


Know and use basic facts about non-Euclidean geometries 


2.26 




2.24 


TRIGONOMETRY 








56 


Rnd trigonometric form of complex numbers and apply DeMoivre's 
Theorem 






2.34 


PROBABILITY AND STATISTICS 








73 


Calculate the expected value of a function of a discrete random variable 


2.09 


2.41 


2.36 


74 


Model an applied problem using mathematical expectation 


2.46 






76 


Explain the consequences of the Central Limit Theorem 


2.08 




2.26 


77 


Use the Central Limit Theorem to calculate probabilities 


2.00 




2.16 


78 


Solve problems using the uniform and chi-square distributions 


1.87 


2.18 


2.08 


79 


Solve continuous probability problems with random variables, etc. 


1.77 


2.11 


2.11 


80 


Solve continuous probability problems with joint and conditional 
probability 


1.94 


2.10 


2.15 


81 


Solve expected value problems for continuous random variables 


1.81 


2.01 


2.05 


82 


Develop test to accept or reject a given null hypothesis 


1.81 


2.43 


1.96 


83 


Discuss sample size, significance level, power, type 1, It error 
relationships 


1.71 


2.18 


1.88 


CALCULUS 








91 


Prove via epsilon-delta that the limit of a function equals the calculated 
value 


2.40 


2.35 


2.42 


102 


Approximate the roots of a function 


2.44 






111 


Evaluate improper integrals 


2.47 






114 


Interchange the order of Integration in double integrals 


1.94 


2.28 


2.20 


115 


Rnd volumes of solids using double integrals 


1.99 


2.41 


2.25 


116 


Determine limits of sequences and simple infinite series 






2.42 


117 


Use tests to show convergence or divergence of series 


2.36 




2.22 


118 


Determine the interval of convergence of a power series 


2.19 


2.41 


2.15 


119 


Determine Taylor series of functions like sin x, cos x, b \ and In x 


2.18 


2.46 


2.15 
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Table 1 (cont.) 







Primary 

Sample 


Supplemental 

Sample 






Teachers 
(N = 180) 


College 
Faculty 
(N = 159) 


Teachers 
(N = 77) 


DISCRETE MATHEMATICS 








123 


Use Laws of AJgebra of Propositions to evaluate equivalence 


2.36 




2.49 


125 


Use Euclidean Algorithm to find greatest common divisor of two 
numbers 






2.49 


126 


Work with numbers expressed In bases other than base ten 


2.34 






127 


Solve modular congruences 


2.05 




2.08 


128 


Prove theorems using modular systems 


1.67 


2.11 


1.92 


129 


Rnd values of functions defined recursivei/ 


2.34 




2.03 


130 


“Translate" between recursive and closed form expressions for a 
function 


2.02 


2.45 


1.90 


131 


Determine if a binary relation is reflexive, symmetric, antisymmetric, 
transitive 


2.21 




2.13 


132 


Determine if a binary relation is an equivalence relation 


2.26 




2.11 


133 


Determine if set is ordered with respect to a binary relation 


2.10 




2.03 


134 


Prove countability of rational numbers and non-countability of real 
numbers 


1.84 


2.18 


1.96 


135 


Use basic terminology in graph theory 


2.15 




2.15 


136 


Identify conditions under which a graph can be traversed 


1.81 


2.29 


1.81 


137 


Know and use the three basic ways of traversing a binary tree 


1.77 


2.03 


1.83 


138 


Solve simple linear programming problems 






2.37 


139 


Rnd and use finite differences of a function 


2.00 


2.42 


2.06 


140 


Interpolate using Newton’s fonvard and backward difference formulas 


1.66 


1.85 


1.77 


ABSTRACT ALGEBRA 








141 


Determine if a set together with an operation is a group 


2.21 




1.93 


142 


Use definition of a group to deduce elementary properties of a group 


2.13 




1.90 


143 


Determine if a set together with two operations is a ring 


1.94 




1.85 


144 


Determine if a set together with two operations is a field 


2.20 




1.85 


145 


Use definition of a field to deduce elementary properties 


2.11 




1.83 


146 


Determine whether mathematical systems are groups, rings, or fields 


2.09 




1.86 


147 


Determine If subset of a group is subgroup or normal subgroup 


1.74 


2.20 


1.81 


LINEAR ALGEBRA 










In a finite dimensional real vector space: 








148 


determine if a subset is a subspace 


1.72 


2.47 


1.89 


149 


determine If a set of vectors is linearly Independent 


1.87 




2.04 


150 


determine if a set of vectors Is basis for vector space 


1.89 




1.90 
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Table 1 (cont.) 







Primary 

Sample 


Supplemental 

Sample 






Teachers 
(N = 180) 


College 
Faculty 
(N * 159) 


Teachers 
(N « 77) 


151 


determine the dimension of the span of a set of vectors 


1.77 


2.34 


1.83 


152 


determine if vectors are orthogonal using the dot product 


1.93 




1.99 


153 


determine effects of linear transformation on a vector space 


1.79 


2.30 


1.92 


154 


determine image and kernel of a linear transformation 


1.54 


2.06 


1.79 


155 


Add, Subtract, and scalar multiply vectors using geometric 
interpretations 






2.38 


157 


Demonstrate understanding and use basic properties of inverses 






2.41 


158 


Determine and apply matrix representation of a linear transformation 


2.27 




2.29 


COMPUTER SCIENCE 








164 


Trace and debug existing computer algorithms 


2.23 




2.36 


165 


Program in two computer languages, one of which is a structured 
language 


1.95 


2.30 


2.16 


MATHEMATICAL REASONING AND MODELING 








170 


Estimate actual and relative error in numerical answer 


2.46 






173 


Determine whether an Isomorphism exists between two mathematical 
systems 


2.13 


2.46 


2.29 


174 


Determine if one mathematical model will describe two different 
situations 


2.37 






175 


Understand the different levels of mathematical impossibility 






2.41 


176 


Use the axiomatic method in modeling and problem solving 


2.43 




2.19 


CONTENT-SPECIFIC PEDAGOGY 








191 


Evaluate impact of learning theorists on mathematics education 


2.04 


2.37 


2.30 
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Those task statements rated less than 2.50 by any of the demographic subgroups mentioned 
above are provided in Table 2. Of the 193 individual task statements, 63 (32.6%) were rated 
below 2.50 by one or more of the subgroups. Similar to the results of the analysis of the 
education professionals, the following dimensions yielded the highest percentages of statements 
with low ratings: Probability and Statistics (10 of 19 statements). Discrete Mathematics (16 of 
21), Abstract Algebra (7 of 7), and Linear Algebra (9 of 12). Of the 63 statements rated below 
the criterion in the demographic analyses, 62 also fell below the criterion for the education 
professionals. In total, 65 of the 193 statements (33.7%) did not meet the 2.50 criterion for 
inclusion by either the education professionals or the demographic subgroups. This leaves 128 task 
statements that can be used (without written rationale) as the foundation for test specifications. 



Correlations of the Profiles of Mean Importance Ratings 

Education professionals . Correlations were computed among arrays of means for the 
teachers and college faculty in the primary sample and the teachers in the supplemental sample. 
The obtained correlations were: Teachers (primary) and Teachers (supplemental), r = .97; 
Teachers (primary) and College Faculty (primary), r = .96; and College faculty (primary) and 
Teachers (supplemental), r = .91. 

Demographic subgroups . Correlations were computed among arrays of means for the 
selected subgroups of respondents (e.g., males and females). This is done as a way of evaluating 
agreement among subgroups. The correlations between the various subgroups are all in the 
upper .90s (see Table 3, page 17). This, combined with the results for the education 
professionals, indicates a high level of agreement among the respondent groups with respect to 
the relative importance of the statements. This is consistent with other findings in the job 
analysis literature (e.g., Schmitt and Cohen, 1989). 



Mean Ratings of Content Coverage 

The survey participants were asked to indicate, using a 5-point rating scale, how well the 
statements within each of the 13 major knowledge categories covered the important aspects of 
the category. Responses to this provide an indication of the adequacy (comprehensiveness) of 
the content domain. The scale values were 1 = Poorly, 2 = Somewhat, 3 = Adequately, 4 = Well, 

5 = Very well. The mean ratings for the teachers and college faculty in the primary sample, the 
teachers in the supplemental sample, and all respondents in the total sample are presented in 
Table 4 (page 17). The overall mean ratings (i.e., for all employment categories in the total 
sample) meet or exceed 4.00 on all categories except Computer Science, which has a moderately 
high rating of 3.87. This supports the notion that the major knowledge categories were 
reasonably well covered and that the overaU content domain was comprehensive. 



Table 2 

Statements Rated Below 2.50 by Demographic Subgroups 



Teaching 

Experience 

(teachers 

only) 


+ 9 


2.37 


2.26 




2.14 


2.47 


2.12 


2.04 


1.90 


1.83 


1.97 


1.85 


1.81 


1,73 


2.42 


2.47 




in 

6 




2.18 


^ 2.42 


2.39 




2.23 


2.10 


2.13 


2.16 


2.25 


2.09 


2.13 


1.94 


2.31 


eg 


2.43 


Geographic Region 


FW 


5? 

cvi 


2.40 




2.25 




2.27 


2.16 


2.06 


1.93 


2.03 


1.94 


in 

p 

oi 


1.91 








CO 








2.28 




2.28 


2.19 


1.92 


52 

p 


2.00 


1.91 


eg 

p 

oi 


1.87 


oi 


in 

eg 




O 


a 

oi 


2.33 




2.31 

1 




2.43 


2.39 


oi 


2.04 1 


2.12 


1.98 


2.13 


2.02 


eg 

eg 






NE 


2.31 


2.34 




1 2.32 




csi 


2.28 


2.15 


2.07 


2.11 


1.96 


2.17 


1.95 


2.40 






li 

UJ 


5 


2.38 


2.39 




2.28 

1 




2.34 


2.25 ’ 


s 

oi 


1 98 


to 

o 

CV- 


1.93 


2.08 


CO 

p 


2.41 






POC 






2.44 


2.34 




2.31 


2.17 


2.02 1 


2.07 


2.15 


in 

p 


2.05 


1.95 


? 

eg 






X 


2 


2.31 


2.31 




2.29 

1 




2.36 


2.26 


2.03 


1.97 


2.04 


1.93 


2.13 


1.89 


2.34 
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Note; F = Female (N = 203); M = Male (N = 328); POC = People of Color (N=43); W = White (N = 483); NE •= Northeast (N=124); C «= Central (N = 138); S - South (N-141); 
FW = Far West (N= 129); 0-5 = 0 to 5 years' teaching experience (N=34); 6+ = 6 or more years' teaching experience (N = 223). 
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Experience 

Race/ (teachers 
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Table 2 (cont.) 
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Table 3 

Correlations of the Mean Importance Ratings Among Demographic Subgroups 





1 


2 3 4 


Sex 


1. Female (N=203) 


1.00 




2. Male (N=328) 


.98 


1.00 


Raclal/Ethnic Background 


1. Minority (N« 43) 


1.00 




2. Majority (N= 483) 


.98 


1.00 


Geographic Region 


1. Northeast (N = 124) 


1.00 




2. Central (N= 138) 


.99 


1.00 


3. South (N- 141) 


.99 


.99 1.00 


4. Far West (N = 129) 


.98 


.99 .99 1.00 


Teaching Experience 
(Teachers only) 


1. 1 - 5 years (N=34) 


1.00 




2. Greater than 5 years (N=223) 


.96 


1.00 



Table 4 

Mean Ratings of Content Coverage 





Primary 

Sample 


Supplemental 

Sample 


Total 

Sample 


Knowledge Category 


Teachers 

(N-180) 


College 

Faculty 

(N-159) 


Teachers 
(N = 77) 


All Employment 
Categories 
(N = 544) 


Arithmetic and Basic Algebra 


4.39 


4.31 


4.49 


4.38 


Geometry 


4.27 


4.04 


4.36 


4.21 


Trigonometry 


4.34 


4.19 


4.42 


4.30 


Functions and Their Graphs 


4.24 


4.12 


4.42 


4.24 


Probability and Statistics 


4.17 


4.06 


4.35 


4.14 


Analytic Geometry 


4.06 


3.91 


4.32 


4.02 


Calculus 


4.31 


4.25 


4.30 


4.32 


Discrete Mathematics 


4.08 


3.96 


4.26 


4.06 


Abstract Algebra 


4.05 


3.86 


4.16 


4.02 


Linear Algebra 


4.05 


3.86 


4.30 


4.00 


Computer Science 


3.96 


3.70 


4.08 


3.87 


Mathematical Reasoning and Modeling 


4.22 


4.03 


4.28 


4.17 


Content-Specific Pedagogy 


4.24 


4.08 


4.23 


4.17 
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Mean Percentage Weights for Test Content Emphasis: Recommendations for Test Content 



In Part III of the survey, Recommendations for Test Content, participants are asked to 
indicate how many test questions (out of 100) should be included from each of the knowledge 
categories. This information may be used by the Advisory Committee to assist them in making 
decisions about how much emphasis the knowledge categories should receive in the test 
specifications. The mean wei^ts for the teachers and college faculty in the primary sample, the 
teachers in the supplemental sample, and all respondents in the total sample are presented in 
Table 5. Arithmetic and basic Algebra and Geometry received the highest average ratings, while 
Abstract Algebra, Linear Algebra, and Discrete Mathematics received the lowest. 



Table 5 

Mean Percentage Weights for Test Content Emphasis 



Primary 

Sample 


Supplemental 

Sample 


Total 

Sample 


Knowledge Category 


Teachers 
(N = 180) 


College 

Faculty 

(N»159) 


Teachers 

(N=77) 


All Employment 
Categories 
(N = 544) 


Arithmetic and Basic Algebra 


17.59 


14.78 


20.53 


17.02 


Geometry 


12.83 


10.97 


13.40 


12.29 


Trigonometry 


9.18 


8.16 


9.87 


8.96 


Functions and Their Graphs 


9.56 


9.12 


7.76 


9.03 


Probability and Statistics 


5.98 


6.80 


6.79 


6.46 


Analytic Geometry 


7.09 


6.39 


5.95 


6.64 


Calculus 


6.89 


8.25 


5.75 


7.21 


Discrete Mathematics 


4.49 


5.53 


3.88 


4.77 


Abstract Algebra 


3.21 


4.18 


2.78 


3.53 


Linear Algebra 


3.75 


4.63 


4.03 


4.14 


Computer Science 


5.39 


5.21 


5.50 


5.32 


Mathematical Reasoning and Modeling 


6.58 


7.31 


5.73 


6-65 


Content-Specific Pedagogy 


7.46 


8.67 


8.03 


7.97 



Summary and Conclusions 

A job analysis was conducted to define a knowledge domain in which newly licensed 
(certified) mathematics teachers must be knowledgeable to perform their jobs in a competent 
manner. A draft domain of important knowledge statements was constructed by ETS Test 
Development staff with expertise in mathematics and ETS Research staff with expertise in job 
analysis. This draft domain was reviewed by an External Review Panel of subject-matter experts 
and revised as they felt necessary. The revised draft was then reviewed, modified, and approved 
by an external Advisory Committee. The revised knowledge domain was then subjected to 
verification/refutation through the use of a national survey of mathematics teachers, teacher 
educators, and state administrators. The survey participants were asked to rate specific task 
statements of the domain using a 5-point importance scale. A cut point of 2.50 (midpoint 
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between moderately important and important) was established to designate task statements as 
eligible (> 2.50) or ineligible (< 2.50) for inclusion in the development of test specifications. 

The results of the mean analysis conducted by teachers and teacher educators indicated that 
64 of 193 task statements were rated less than 2.50. This represents 33.2% of the content 
domain. When the same analysis was conducted for demographic subgroups, very similar results 
were obtained. In total, 65 of the 193 statements (33.7%) did not meet the 2.50 criterion for 
inclusion in these two analyses. 

The 128 task statements that were verified to be important by those surveyed should be used as 
the foundation for the development of test specifications. Test specifications that are linked to the 
results of a job analysis provide support for the content validity of the derived assessment 
measures and may be considered part of an initial step in ensuring the fairness to subgroups of 
mathematics teacher candidates of the derived assessment measures. It is reasonable to assume 
that because of testing and psychometric constraints (e.g., time limits, ability to measure content 
reliably) not all of the verified content will be included in the new assessment measure. One 
source of information that may be used to guide the Advisory Committee in their decision as to 
what verified content to include is the mean importance rating. Although a rank ordering of the 
content by mean importance rating is not implied, it is recommended that initial consideration 
be given to content that is well above the criterion and represents the appropriate breadth of 
content coverage as stipulated in the test specifications. 

The computation of correlation coefficients to assess relative agreement in terms of 
perceived importance of the task statements revealed a very high level of agreement. All 
coefficients exceeded .90. These findings indicate that there is substantial agreement in the 
importance ratings given by a diverse group of education professionals. 

Evidence was also provided in this study of the comprehensiveness of the content domain 
within each of the 13 major knowledge categories. The results indicated that the survey 
respondents thought the categories were reasonably well covered by their task statements. 

Finally, we collected data in the Recommendations fc.' Test Content section of the survey 
regarding the emphasis that should be given in the test to each of the 13 categories. This 
information will be used by the Advisory Committee in their decisions about the appropriate 
weighting of the test. 

In summary, this study took a multi-method approach to identify a content domain that is 
related to the job of the newly licensed mathematics teacher. The job analysis process allowed 
for input from many practicing professionals in mathematics education. The results of the study 
will be used to develop specifications for the mathematics test that will be included as part of 
the subject assessments of The Praxis Series: Professional Assessments for Beginning 
Teachers™. 
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Appendix A 
External Review Panel 




A-1 




Deborah Ball 

Assistant Professor of Teacher Education 
Michigan State University 
East Lansing, MI 

Ms. Ball received her Ph.D. in Educational Policy from Michigan State University. She 
has taught elementary school for 13 years and college for 2 years. Currently, she teaches 
Preservice Mathematics, Introduction to Education, and a Doctoral Seminar on Disciplinary 
Knowledge. She has received an Excellence in Teaching Award and the Outstanding 
Dissertation Award while at Michigan State. 



Steven Conrad 
Roslyn High School 
Roslyn Heights, NY 

Mr. Conrad has an M.S. degree from Yeshiva University. He has been a high school 
mathematics teacher for 23 years. The courses he typically teaches are Geometry, Algebra II 
and Trigonometry, Pre-Calculus, Calculus, and AP Calculus. He is a member of the American 
Federation of Teachers (AFT) and has received the Presidential Award for his teaching. 



Ms. Gloria Donaldson 

Oklahoma School of Science and Math 

Oklahoma City, OK 

Ms. Donaldson received an M.A.T. degree from the University of Alabama. She is a 
Charter Mathem.atics Instructor with 24 years of teaching experience. The courses she has 
taught range from seventh grade math to Calculus. She was awarded the Andalusia Chamber of 
Commerce Teacher of the Year award and the Richover Faculty Award and was a nominee for 
the Presidential Award. She has twice been the President of Alabama Teachers of Mathematics 
and has served on the Johns Hopkins Center for the Advancement of Academically Talented 
Youth. 



John Dossey 

Distinguished University Professor of Mathematics 
Illinois State University 
Normal, IL 

Mr. Dossey received his Ph.D. from the University of Illinois. He has taught courses 
such as Problem Solving, Number Theory, and History of Mathematics for 24 years. He is a 
Past President of the National Council of Teachers of Mathematics. He has served on the 
Board of Governors for the Mathematics Association of America and the Executive Committee 
of the Mathematics/Science Board of the National Research Council. 



Shirley Frye 

Director of Curriculum and Instruction 
Scotsdale Schools 
Scotsdale, AZ 

Ms. Frye has an M.A. degree from Arizona State University. She is currently the 
Director of Curriculum and Instruction for Scotsdale Schools. 



Jim Gates, Executive Director 

National Council of Teachers of Mathematics 

Reston, VA 

Mr. Gates received a Ph.D. from George Washington University. In addition to being 
the Executive Director of the National Council of Teachers of Mathematics, he has 20 years of 
teaching experience at the junior high, senior high, and college levels. He has been given the 
National Science Foundation Institute Award, the General Electric, Institute Award, and the 
General Motors Award for his contributions. 



Linda Hackett 
Foxcroft School 
Middleburg, VA 

Ms. Hackett has been a secondary school mathematics instructor for 10 years. She 
typically teaches AP Calculus, Pre-Calculus, and Algebra II. She received her M.Ed. from 
Edinborough University. 



Michael Lambe 
Grossmont Union High School 
Spring Valley, CA 

Mr. Lambe received his M.S. from the State University of New York at Buffalo. He has 
taught secondary school mathematics for 23 years. In addition to being the Department Head 
of Mathematics at Grossmont Union, Mr. Lambe typically teaches Intermediate Algebra, Pre- 
Calculus, Analysis, and Trigonometry. He has served on the Commission on Teacher 
Credentialing and the California Mathematics Council. 



James R. Leitzel 

Associate Professor and Vice Chairman 
Department of Mathematics 
The Ohio State University 
Columbus, OH 

Mr. Leitzel received his Ph.D. from Indiana University. He has been teaching 
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TASK ANALYSIS INVENTORY 



FOR MATHEMATICS TEACHERS 



By 



Educational Testing Service 
Princeton, New Jersey 



Copyright ® 1990 by Educational Testing Service. All rights reserved. 



INTRODUCTION 



Educational Testing Service (ETS) is developing a new generation of assessments for the purpose of 
licensing teachers. The inventory that follows is part of our development effort and is designed to gather 
information concerning the job of the secondarv^school mathematics teacher. It was developed by high 
school mathematics teachers, college mathematics professors, mathematics teacher educators, and state 
department of education officials working with ETS staff. 

The educators who collaborated on the construction of this questionnaire recognize that mathematics 
teachers in secondary school may be required to teach a variety of courses ranging from remedial arithmetic 
to calculus, and they will have to teach students with varying levels of ability and background. For these 
reasons, the collaborators believe that mathematics teachers should have a broad and deep understanding of 
many different areas of mathematics in order to integrate the subject matter and to teach it in a meaningful 
way. Part I of the inventory asks you to react to a list of tasks and to rate each task with respect to its 
importance for a newlvdicensed (certified^ mathematics teacher. Try not to relate each task to your own job, 
but rather to what you believe a new teacher should know and be able to do in order to be a competent 
mathematics teacher at any level of secondary school. Part II of the inventory asks that you make 
recommendations concerning the content of the new test. Finally, Part III asks you for some background 
data. The information you provide will ultimately be used to guide the development of the successor to the 
National Teacher’s Examination for Mathematics. It is expected that the new test will differ from the 
current examination in both content and design. In addition to the development of a new test, this study 
represents an important contribution in its own right to our understanding of mathematics teaching as a 
profession. We expect the results of the study to be widely disseminated and to have ramifications for 
teacher education and development. 

The inventory has been mailed to a carefully selected sample of 800 professionals like you. The value of 
a survey like this one is directly related to the number of individuals who complete and return their 
questionnaires. Because you represent a large number of professionals, your responses are very important to 
us. Please take the time to complete the inventory. 
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PART I - INVENTORY OF TASKS 



The purpose of Part I is to ascertain what you believe a new teacher should know and be able to do in order 
to be a competent mathematics teacher. On the follovnng pages you will find thirteen broad categories (e.g., 
arithmetic and basic algebra, geometry, calculus, mathematical reasoning and modeling, etc.). Beneath each 
category is a list of task statements related to that category. The categories are based on mathematical 
topics rather than typical courses. Because of this, some tasks that are taught in a particular course may 
appear in a different category from the name of that course. 

For each statement in this section, you will be asked to make the following judgment: 



IMPORTANCE: How important is it that a newlv-licensed ('certified') mathematics teacher be 
able to perform this task in a competent manner? 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



To familiarize yourself with the categories and task statements you may wish to briefly glance through Part I 
before you make your rating judgments. 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



A. ARITHMETIC AND BASIC ALGEBRA IMPORTANCE 

1. Order any finite set of real numbers and recognize equivalent forms of a 

number (e.g., 1/10, 0.1, 10'^ 0 1 2 3 4 

2. Use numbers in a way that is most appropriate in the context of a problem 
(e.g., appropriately rounded numbers, numbers written in scientific notation, 

using 5(100 - 1) for 5(99), etc.) 0 1 2 3 4 

3. Demonstrate an understanding of the properties of counting numbers (e.g., 

prime or composite, even or odd, factors, multiples) 0 1 2 3 4 

4. Add, subtract, multiply, and divide rational numbers expressed in 

any form 0 1 2 3 4 

5. Apply the order of operations to problems involving addition, subtraction, 
multiplication, division, roots, and powers, with and without grouping 

symbols 0 1 2 3 4 

6. Interpret and apply the concepts of ratio, proportion, and percent in 

appropriate situations 0 1 2 3 4 

7. Solve problems that involve measurement in the metric system 0 1 2 3 4 

8. Solve problems that involve measurement in the traditional system 0 1 2 3 4 

9. Find positive integral powers and roots of perfect powers (e.g., what is the 

cube root of 9261) 0 1 2 3 4 

10. Estimate values of expressions involving decimals, exponents, and radicals 

(e.g., what is a good estimate for the square root of 21) 0 1 2 3 4 

11. Solve problems involving average, including arithmetic mean and weighted 

average 0 1 2 3 4 

12. Evaluate algebraic expressions and formulas (e.g.. If x = 5 and y = 6, what 

is the value of + 5y ?) 0 1 2 3 4 

13. Solve for a selected variable in a literal equation or formula (e.g.. If A = 

bh/2, then h = 2A/b) 0 1 2 3 4 

14. Classify a number as a rational, irrational, real, and/or complex 0 1 2 3 4 

15. Identify the properties (e.g., closure, commutativity, associativity, 
distributivity) of the basic operations (i.e., addition, subtraction, 

multiplication, division) on the standard number systems 0 1 2 3 4 
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Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newlv-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 



0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



A. ARITHMETIC AND BASIC ALGEBRA (continued) IMPORTANCE 

16. Given newly defined operations on a number system, determine whether the 

closure, commutative, associative, or distributive properties hold 0 1 2 3 4 

17. Identify the additive and multiplicative mverses of a number 0 1 2 3 4 

18. Add, subtract, and multiply polynomials 0 1 2 3 4 

19. Divdde polynomials 0 1 2 3 4 

20. Add, subtract, multiply, and divide algebraic fractions 0 1 2 3 4 

21. Translate verbal expressions and relationships into algebraic expressions or 

equations 0 1 2 3 4 

22. Solve and graph linear equations and inequalities in one or two variables . . 0 12 3 4 

23. Solve and graph systems of linear equations and graph inequalities in two 

variables 0 1 2 3 4 

24. Solve and graph nonlinear algebraic equations and graph inequalities 0 12 3 4 

25. Perform standard algebraic operations involving complex numbers, radicals, 

and exponents, including fractional and negative exponents 0 1 2 3 4 

26. Determine any term of a binomial expansion using Pascal’s triangle or some 
other method (e.g., What is the fourth term of the binomial expansion of 

(a + b)"?) 0 1 2 3 4 

27. Solve equations and inequalities involving absolute values (e.g., |x + 2| =6, 

|x + 2| + 3x = 6) 0 1 2 3 4 

28. Present geometric interpretations of algebraic principles (e.g., the triangle 

inequality and the distributive principles). See example 0 1 2 3 4 

4 + 3 

* m * * ««« **** *** 

«««« * * * * * * * * * * 



5(4+3) = (5x4) + (5x3) 
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Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newlv-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



R GEOMETRY IMPORTANCE 

29. Use relationships (e.g., congruency, similarity) among two-dimensional 

geometric figures to solve problems 0 1 2 3 4 

30. Use relationships (e.g., congruency, similarity) among three-dimensional 

geometric figures to solve problems 0 1 2 3 4 

31. Solve problems involving the properties of parallel and perpendicular 

lines Q 1 2 3 ^ 

32. Solve problems using special triangles, such as isosceles and equilateral .... 01234 

33. Solve problems using the relationships of the parts of triangles, such as sides, 

angles, medians, midpoints, and altitudes 0 1 2 3 4 

34. Apply the Pythagorean Theorem to solve problems 0 1 2 3 4 

35. Solve problems using the properties of special quadrilaterals, such as the 

square, rectangle, parallelogram, rhombus, and trapezoid 0 1 2 3 4 

36. Describe relationships among sets of special quadrilaterals, such as the 

square, rectangle, parallelogram, rhombus, and trapezoid 0 1 2 3 4 

37. Solve problems using the properties (e.g., angles, sum of angles, number of 

diagonals, and vertices) of polygons with more than four sides 0 1 2 3 4 

38. Solve problems using the properties of circles, including those involving 
inscribed angles, central angles, chords, radii, tangents, secants, arcs, and 

sectors 0 1 ^ ^ ^ 

39. Compute the perimeter and area of triangles, quadrilaterals, and circles, and 

regions that are combinations of these figures 0 1 2 3 4 

40. Compute the surface area and volume of right prisms, pyramids, cones, 

cylinders, and spheres, and solids that arc combinations of these figures ... 01234 

41. Solve problems involving reflections, rotations, and translations of points, 

lines, or polygons in the plane 0 1 2 3 4 

42. Execute geometric constructions using straight-edge and compass (e.g., bisect 

an angle, erect a perpendicular) 0 1 2 3 4 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a n ewiv *) icensed (certified") mathematics teacher be able to perform the task in a competent mzinner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



B. GEOMETRY (continued) IMPORTANCE 

43. Prove that a given geometric construction yields the desired result 0 1 2 3 4 

44. Know and use basic facts about non-Euclidean geometries and the 
ramifications of the three postulates pertaining to the existence of parallel 

lines 0 1 2 3 4 

C. TRIGONOMETRY 

45. Define and use the six basic trigonometric relations on the right triangle ... 01234 

46. Solve problems involving right triangles which have angles of 30° or 45° 

without tables or calculators 0 1 2 3 4 

47. Identify the relationship between radian measures and degree measures of 

angles 0 1 2 3 4 

48. Apply the law of sines and the law of cosines in the solution of problems . . 0 12 3 4 

49. Define and use the six basic trigonometric functions as defined on the unit 

circle using radian measure 0 1 2 3 4 

50. Solve problems involving trigonometric functions evaluated at such numbers 

as TT, tt/6, 2tt/3, 97t/ 4, and -7 t/3 0 1 2 3 4 

51. Recognize the graphs of the six basic trigonometric functions and identify 

their period, amplitude, phase displacement or shift, and asymptotes 0 1 2 3 4 

52. Apply the formulas for the trigonometric functions of x/2, 2x, x + y, and 

X - y in terms of the trigonometric functions of x and y 0 1 2 3 4 

53. Prove identities using the basic trigonometric identities 0 1 2 3 4 

54. Solve trigonometric equations and inequalities 0 1 2 3 4 

55. Given a point in the rectangular coordinate system, identify the 

corresponding p>oint in the polar coordinate system 0 1 2 3 4 

56. Find the trigonometric form of complex numbers and apply DcMoivre’s 

Theorem 0 1 2 3 4 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licensed (certified) mathematics teacher be able to perform the task m a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



D. FUNCTIONS AND THEIR GRAPHS IMPORTANCE 

57. Understand function notation for functions of one variable and be able to 
work with the algebraic definition of a function (i.e., for every x there is at 
most one y) and be able to identify whether a graph in the plane is the 

graph of a function 0 1 2 3 4 

58. Given a graph, select from a list the most appropriate equation for the 

graph 0 1 2 3 4 

59. Given an equation, graph it 0 1 2 3 4 

60. Use the definition of a function as a mapping and be able to work with 

functions given in this way (e.g., f: ( x, y ) -> ( + y^, x^- y^ )) 0 1 2 3 4 

61. Find the domain and/or range of a function 0 1 2 3 4 

62. Use the properties of algebraic, trigonometric, logarithmic, and exponential 
functions to solve problems (e.g., finding composite functions and inverse 

functions) 0 1 2 3 4 

63. Find the inverse of a one-to-one function in simple cases and know why one- 

to-one functions have inverses 0 1 2 3 4 

64. Determine the graphical properties and sketch a graph of a linear, step, 
absolute value, or quadratic function (e.g., slope, intercepts, intervals of 

increase or decrease, axis of symmetry) 0 1 2 3 4 

E. PROBABILITY AND STATISTICS 

65. Organize data into a presentation that is appropriate for solving a problem 

(e.g., construct a histogram and use it in the calculation of probabilities) ... 01234 

66. Solve probability problems involving finite sample spaces by actually counting 

outcomes appropriately 0 1 2 3 4 

67. Solve probability problems using counting techniques (e.g.. If 3 cards are 

drawn from a standard deck of 52 cards, what is the probability that all 3 are 0 12 3 4 
aces?) 

68. Solve probability problems involving independent trials (e.g.. If a coin is 

tossed 5 times, what is the probability that at least 3 heads occur?) 0 1 2 3 4 



Using the scale below, please circle the number that best desaibes your judgment as to how important it is that 
a newlv-liccnsed Ccertified’i mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



E. PROBABILITY AND STATISTICS (CONTINUED) IMPORTANCE. 

69. Solve problems using the binomial distribution and be able to determine 

when the use of the binomial distribution is appropriate 0 1 2 3 4 

70. Solve problems involving joint probability 0 12 3 4 

71. Find and interpret common measures of central tendency (population mean, 
sample mean, median, mode) and know which is the most meaningful to use 

in a given situation 0 ^ ^ ^ ^ 

72. Find and interpret common measures of dispersion (range, population 
standard deviation, sample standard deviation, population variance, sample 

variance) i ^ ^ ^ ^ ^ 

73. Calculate the expected value of a function of a discrete random variable 
given the probability distribution function of the random variable or its 

expected value (e.g.. If E(X) = 2, what is E(2X + 2)?) 0 1 2 3 4 

74. Model an applied problem by using the mathematical expectation of an 
appropriate discrete random variable (c.g., fair coins, expected winnings, 

expected profit) 0 1 ^ ^ ^ 

75. Solve problems using the normal distribution 0 1 2 3 4 

76. Explain the consequences of the Central Limit Theorem and why it 

establishes the importance of the normal distribution in the study of statistics 0 12 3 4 

77. Use the Central Limit Theorem to calculate probabilities (e.g., A random 
sample of 100 observations is selected from a population with /x = 30 and 

o = 16. Approximate the probability that 5T < 28.) 0 1 2 3 4 

78. Solve problems using the uniform and chi-square distributions 0 1 2 3 4 

79. Solve problems involving continuous probability using such concepts as 

random variables, probability density function, and cumulative distribution 0 12 3 4 

function 

80. Solve problems involving continuous probability using such concepts as joint 

and conditional probability 0 1 2 3 4 

81. Solve expected value problems for continuous random variables 0 1 2 3 4 

82. Develop a test to determine whether to accept or reject a given null 

hypothesis, Hq.. 0 ^ 2 3 4 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newlv-licensed (certified'! mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



E. PROBABILITY AND STATISTICS (CONTINUED) IMPORTANCE 

83. Discuss the concepts of and relationships between sample size, level of 

significance, power, and type I (a) and type II errors 0 1 2 3 4 

F. ANALYTIC GEOMETRY 

84. Determine the equations of lines and planes given appropriate information . 0 12 3 4 

85. Make calculations in 2-spacc or 3-space (e.g., distance between two points, 
the coordinates of a midpoint of a line segment, distance between a point 

and a plane) 0 1 2 3 4 

86. Given a geometric definition of a conic section^ derive the equation for the 
conic section (e.g., given that a parabola is the set of points that are 

equidistant from a given point and a given line, derive its equation) 0 1 2 3 4 

87. Determine which conic section is represented by a given equation if the axis 

of symmetry is parallel to one of the coordinate axes 0 1 2 3 4 

88. Determine which conic section is represented by a given equation if no 

restrictions are placed on its location in the plane 0 1 2 3 4 

G. CALCULUS 

89. Discuss informally what it means for a function to have a limit at a point . . 0 12 3 4 

90. Calculate limits of functions or determine that the limit does not exist .... 01234 

91. Prove via an epsilon-delta proof that the limit of a function is actually equal 

to the calculated value 0 1 2 3 4 

92. Solve problems using the properties of limits 

(e.g., lim f(x) + Urn g(x) = lim (f(x) + g(x)) 0 1 2 3 4 

X-K X“»C X“»C 

93. Use limits to show that a particular function is continuous 0 1 2 3 4 

94. Use L’Hopitafs rule, where applicable, to calculate limits of functions .... 01234 

95. Relate the derivative of the function to a limit or to the slope of a curve ... 01234 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licensed (certified) mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



G. CALCULUS (continued) 

96. Explain conditions under which a continuous function does not have a 

derivative 

97. Differentiate algebraic expressions, trigonometric functions, and exponential 
and logarithmic functions using the sum, product, quotient, and chain rules . 

98. Use implicit differentiation 

99. Make numerical approximations of derivatives and integrals 

100. Use differential calculus to analyze the behavior of a function (e.g., find 

relative maxima £md minima, concavity) 

101. Use differential calculus to solve problems involving related rates and rates 

of change 

102. Approximate the roots of a function (e.g., using Newton’s method with 

derivatives) 

103. Use differential calculus to solve applied minima-maxima problems 

104. Solve problems using the Mean Value Theorem of differential calculus . . . . 

105. Explain the significance of, and solve problems using, the Fundamental 

Theorem of Calculus 

106. Demonstrate an intuitive understanding of the process of integration as 

finding areas of regions in the plane through a limiting process 

107. Integrate functions using algebraic substitutions 

108. Integrate functions using **integration by parts" 

109. Integrate functions using trigonometric substitutions 

110. Integrate logarithmic and exponential functions 

111. Evaluate improper integrals 

112. Use integral calculus to calculate the area of regions in the plane 

113. Calculate the volume of solids formed by rotating plane figures about 

a line 



IMPORTANCE 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 



0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



G. CALCULUS (continued) 

114. Interchange the order of integration in double integrals 

115. Find volumes of solids using double integrals 

116. Determine the limits of sequences and simple infinite series 

117. Use standard tests to show convergence (either conditional or absolute) or 

divergence of series (e.g., comparison, ratio, etc.) 

118. Determine the interval of convergence of a power series 

119. Determine the Taylor series of functions such as sin x, cos x, e* and In x . . 

H. DISCRETE MATHEMATICS 

120. Use the basic terminology and, given the definitions, use the symbols of logic 

121. Solve problems involving the union and intersection of sets, subsets and 

disjoint sets 

122. Use truth tables to verify statements 

123. Use Laws of Algebra of Propositions to evaluate equivalence of complex 

logical expressions (e.g., De Morgan’s Laws) 

124. Solve basic problems involving permutations and combinations 

125. Use the Euclidean Algorithm to find the greatest common divisor of two 

numbers 

126. Work with numbers expressed in bases other than base ten 

127. Solve modular congruences 

128. Prove theorems using modular systems 

129. Find values of functions defined recursively 

130. "Translate" between recursive and closed form expressions for a function . . 



IMPORTANCE 
0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
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Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licensed (certified') mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



H. DISCRETE MATHEMATICS (continued) 

131. Determine if a binary relation on a set is reflexive, symmetric, antisymmetric, 

or transitive 

132. Determine if a binary relation on a set is an equivalence relation 

133. Determine if a set is ordered with respect to a binary relation 

134. Prove the countability of the rational numbers and non-countability of the 

real numbers 

135. Use basic terminology in graph theory (e.g., graphs, trees, binary trees) . . . 

136. Identify the conditions under which a graph can be traversed in such a way 

that each edge is traversed exactly once and the conditions under which each 
vertex is visited exactly once 

137. Know and use the three basic ways of traversing a binary tree, and use a 

binary tree to represent an algebraic expression 

138. Solve simple linear programming problems 

139. Find and use finite differences of a function 

140. Interpolate using Newton’s forward (advancing) or backward difference 

formulas 

I. ABvSTRACT ALGEB RA 

141. Determine if a particular set together with a given operation is a group . . . 

142. Use the definition of a group to deduce elementary properties of a group . . 

143. Determine if a particular set together with two operations is a ring 

144. Determine if a particular set together with two operations is a field 

145. Use the definition of a field to deduce elementary properties of a field .... 

146. Determine whether various mathematical systems (e.g., rational numbers, 
matrices, motions of an equilateral triangle) are groups, rings, or fields . . . . 

147. Determine if a particular subset of a group is a subgroup or a normal 

subgroup 



IMPORTANCE 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 ^ 
0 12 3 4 

0 12 3 4 

0 12 3 4 



Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newlvdicensed (certified"^ mathematics teacher be able to perform the task in a competent manner. 



EMC 



IMPORTANCE 



0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



J. LINEAR ALGEBRA 



IMPORTANCE 



In a finite dimensional real vector space: 

148. determine if a subset is a subspace 

149. determine if a set of vectors is linearly independent 

150. determine if a set of vectors is a basis for the vector space 

151. determine the dimension of the span of a set of vectors 

152. determine if vectors are orthogonal using the dot product 

153. determine the effects of a linear transformation on a vector space 

154. determine the image and kernel of a linear transformation 

155. Add, subtract, and scalar multiply vectors using geometric interpretations of 

these operations and use in real world applications 

156. Scalar multiply, add, subtract, and multiply matrices 

157. Demonstrate an understanding and use the basic properties of inverses of 

matrices 

158. Determine and apply the matrix representation of a linear transformation . . 

159. Use matrix techniques to solve systems of linear equations 



0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 



K. COMPUTER SCIENCE 



160. Demonstrate an understanding of the roles of the hardware and software 

components of a computer system (e.g., output devices, CPU, disks, 
operating systems, secondary storage devices) 

161. Know basic computer terminology (e.g., files, I/O, records) 

162. Use "user friendly” software (e.g., classroom instruction packages, graphics 

software, spreadsheets) 

163. Develop computer algorithms to solve mathematical problems 



0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 



Ob 
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Using the scale below, please circle the number that best describes your judgment as to how im portanl it is that 
a newly-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



K. COMPUTER SCIENCE (continued) 

164. Trace and debug eristing computer algorithms 

165. Program in two computer languages, at least one of which is a structured 

language (e.g., FORTRAN, PASCAL) 

L. MATHEMATICAL REASONING AND MODELING 

166. Demonstrate an understanding of a physical situation or a verbal description 

of a situation and develop a mathematical model of it 

167. Determine appropriate mathematical strategies to solve a problem. These 

strategies might include conjectures, counterexamples, inductive reasoning, 
deductive reasoning (mathematical induction, proof by contradiction, direct 
proof, and other types of proof), and deciding which tools are appropriate 
(e.g., discussion with others, mental math, pencil and paper, calculator, 
computer, trees and graphs, fingers) 

168. Recognize the reasonableness of results ^ven the context of a problem .... 

169. Using estimation, test the reasonableness of results 

170. Estimate the actual and relative error in the numerical answer to a problem 

by analyzing the effects of roundoff and truncation errors introduced in the 
course of solving a problem 

171. Having solved a problem, reconsider the strategics used. Are there other 

appropriate strategies? Which strategics are the most efficient? Can these 
strategies be used to solve other problems? Can these strategies be used to 
prove a more general result? 

172. Communicate rCvSults in an appropriate form (e.g., correct English sentences, 

tables, charts, graphs) 

173. Determine wliether an isomorphism exists between two mathematical 

systems 

174. Determine whether one mathematical model will describe two apparently 

different situations 




IMPORTANCE 
0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
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Using the scale below, please circle the number that best describes your judgment as to how important it is that 
a newly-licenscd (certified) mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



L. MATHEMATICAL REASONING AND MODELING (continued) IMPORTANCE 

175. Demonstrate an understanding of the different levels of mathematical 
impossibility, such as: I lack the mathematical skills to do it; No one has 
been able to do it as yet (e.g., prove Goldbach’s conjecture or Fermat’s last 
theorem); No one will ever be able to do it (e.g., trisect a general angle with 

straight edge and compass) 0 1 2 3 4 

176. Use the axiomatic method in modeling and problem solving 0 1 2 3 4 

M. CONTENT«SPECIFIC PEDAGOGY 

In addition to content, mathematics teachers also need to know something about 
pedagogy specific to mathematics. Please note in the following statements that the 
phrase ” a particular group of students'' is not meant to exclude activities and tasks 
aimed at teaching an individual student 

111. Evaluate a given scope and sequence of mathematics topics 0 1 2 3 4 

178. Develop a scope and sequence of a mathematics topic for a particular level 

and justify it 0 1 2 3 4 

179. Given an example of a student’s work that contains an error arising from a 
misconception, identify the misconception and suggest methods for 

correcting it 0 1 2 3 4 

180. Identify the prerequisite knowledge and skills that students ought to have 

before being taught a particular topic - 0 1 2 3 4 

181. Develop questions that ask students to display their current level of 

understanding of a particular topic 0 1 2 3 4 

182. Given a particular problem, identify several problem-solving strategics (e.g., 
guess and check, reduce to a simpler problem, draw a diagram, work 

backwards) that might assist students to solve the problem 0 1 2 3 4 

183. Use appropriate forms of representation (e.g., analogies, drawings, examples, 
symbols, manipulativcs) for mathematics subject matter for a particular 
group of students that help make mathematics understandable and 

interesting 0 1 2 3 4 

184. Use a variety of teaching strategics (e.g., laboratory work, supervised 
practice, group work, lecture) in mathematics appropriate for a particular 

group of students and a particular topic 0 1 2 3 4 

185. Integrate concepts to show relationships among topics 0 1 2 3 4 



Using the scale below> please circle thfe number that best describes your judgment as to how importanl it is that 
a newlv-licensed (certified^ mathematics teacher be able to perform the task in a competent manner. 



IMPORTANCE 

0 Of no importance 

1 Of little importance 

2 Moderately important 

3 Important 

4 Very important 



M. CONTENT^SPECIFIC PEDAGOGY (continued) 

186. Relate mathematical concepts and ideas to real-world situations 

187. Identify, evaluate, and use curricular materials and resources for 

mathematics (e.g., textbooks and other printed materials, computer software, 
base 10 blocks, geoboards, egg cartons, etc.) in ways appropriate for a 
particular group of students and a particular topic 

188. Know procedures for controlling the social atmosphere of a classroom 

without restricting divergent mathematical thought 

189. Know how society is affected by its general level of mathematical knowledge 

and know how societal influences differentially affect the mathematics 
education of gender, racial, ethnic, and socio-economic groups 

190. Know how to use information about different gender, racial, ethnic, and 
socio-economic groups to enhance these groups’ learning of mathematics . . 

191. Evaluate the impact of learning theorists (e.g., Dewey, Piaget, Pestalozzi, the 

Van Hides, Montessori, Thorndike) on mathematics education 

192. Identify, evaluate, and use appropriate evaluation strategies (e.g., 

observations, interviews, oral discussions, written tests) to assess student 
progress in mathematics 

193. Write specific evaluation items to test for specific mathematical skill . . . . 




^ J 



IMPORTANCE 
0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 



CONTENT COVERAGE 



The purpose of this section is to obtain your judgment of how well this inventory covered the important tasks 
a newly-licensed fcertifiedl mathematics teacher should be able to perform. We are interested in obtaining 
separate judgments for each of the thirteen categories included in the inventory. 

Please circle the number on the rating scale that best represents your judgment. 



How well did the tasks cover the important aspects of this category? 


1 


Poorly 


2 


Somewhat 


3 


Adequately 


4 


WeU 


5 


Very well 



194. ARITHMETIC AND BASIC ALGEBRA 1 2 3 4 5 

195. GEOMETRY 1 2 3 4 5 

196. TRIGONOMETRY 1 2 3 4 5 

197. FUNCTIONS AND THEIR GRAPHS 1 2 3 4 5 

198. PROBABILITY AND STATISTICS 1 2 3 4 5 

199. ANALYTIC GEOMETRY 1 2 3 4 5 

200. CALCULUS 1 2 3 4 5 

201. DISCRETE MATHEMATICS 1 2 3 4 5 

202. ABSTRACT ALGEBRA 1 2 3 4 5 

203. LINEAR ALGEBRA 1 2 3 4 5 

204. COMPUTER SCIENCE 1 2 3 4 5 

205. MATHEMATICAL REASONING AND MODELING 1 2 3 4 5 

206. CONTENT-SPECIFIC PEDAGOGY 1 2 3 4 5 



CO 
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207. Please use this space to list any important categories that you feel were not included m the inventory 
(see prewous page for a listing of the thirteen categories) 



208. Please use this space to list any important task statements that you feel were not included within the 
thirteen categories. For each statement please indicate the category to which it belongs. 
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PART II - RECOMMENDATIONS FOR TEST CONTENT 



Here are the thirteen topics covered in Part I of this inventory. If a licensing examination for mathematics 
teachers contained 100 questions, how many questions should be included from each topic? If you feel a 
topic should not be included in the exam, put 0 in the space provided. Make sure your responses sum to 
100 . 

GENERAL TOPICS NUMBER OF TEST QUESTIONS 

(out of 100) 

209. ARITHMETIC AND BASIC ALGEBRA 

210. GEOMETRY 

211. TRIGONOMETRY 

212. FUNCTIONS AND THEIR GRAPHS 

213. PROBABILITY AND STATISTICS 

214. ANALYTIC GEOMETRY 

215. CALCULUS 

216. DISCRETE MATHEMATICS 

217. ABSTRACT ALGEBRA 

218. LINEAR ALGEBRA 

219. COMPUTER SCIENCE 

220. MATHEMATICAL REASONING AND MODELING 

221. CONTENT-SPECinC PEDAGOGY 



TOTAL 



100 




PART III - BACKGROUND INFORMATION 



The information which you provide in answering the questions in this section is completely confidential and 
will be used for research purposes only. Your responses will be grouped statistically with those of other 
individuals who are participating in this survey. A vital part of the statistical analysis consists of grouping 
people with varying experience and varying backgrounds. To do this, we need your answers to the following 
questions. 



222. In which state do you work? 



1. Alabama 


18. Kentucky 


36. Ohio 


2. Alaska 


19. Louisiana 


37. Oklahoma 


3. Arizona 


20. Maine 


38. Oregon 


4. Arkansas 


21. Maryland 


39. Pennsylvania 


5. California 


22. Massachusetts 


40. Rhode Island 


6. Colorado 


23. Michigan 


41. South Carolina 


7. Connecticut 


24. Minnesota 


42. South Dakota 


8. Delaware 


25. Mississippi 


43. Tennessee 


9. District of 


26. Missouri 


44. Texas 


Columbia 


27. Montana 


45. Utah 


10. Florida 


28. Nebraska 


46. Vermont 


11. Georgia 


29. Nevada 


47. Virginia 


12. Hawaii 


30. New Hampshire 


48. Washington 


13. Idaho 


31. New Jersey 


49. West Virginia 


14. Illinois 


32. New Mexico 


50. Wisconsin 


15. Indiana 


33. New York 


51. Wyoming 


16. Iowa 


34. North Carolina 




17. Kansas 


35. North Dakota 





223. What is your age? 

1. Under 25 

2. 25-34 

3. 35-44 

4. 45-54 

5. 55-64 

6. Over 65 



224. What is your sex? 

1. Female 

2. Male 



O 
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225. How do you describe yourself? 

1. American Indian, Eskimo, or Aleut 

2. Black or African American 

3. Mexican American or Chicano 

4. Oriental or Asian American 

5. Puerto Rican 

6. Other Hispanic or Latin American 

7. White 

8. Other 

226. Which of the following best describes your highest educational attainment? 

1. Less than a Bachelor's degree 

2. Bachelor's degree 

3. Bachelor's degree + additional credits 

4. Master's degree 

5. Master's degree + additional credits 

6. Doctorate 

227. Which of the following best describes your current employment status? 

1. Temporary substitute (assigned on a daily basis) 

2. Permanent substitute (assigned on a longer term basis) 

3. Regular teacher (not a substitute) 

4. Principal or Assistant Principal 

5. School Administrator 

6. Curricular Supervisor 

7. State Administrator 

8. College Faculty 

9. Other (please specify) 

228. How many years have you taught mathematics? 

1. Less than a year 

2. 1-2 years 

3. 3 - 5 years 

4. 6-10 years 

5. 11 - 15 years 

6. 16-20 years 

7. 21 or more years 

8. Never taught mathematics 

229. What grade level(s) are you currently teaching? (Circle ^that apply) 

1. Preschool/Kindcrgarten 

2. Grades 1-4 

3. Grades 5-8 

4. Grades 9-12 

5. College 

6. Do not currently leach 

7. Other (please specify) 
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230. Which of the followng best describes your current teaching assignment? (Circle all that apply) 



1. Arithmetic and Basic Algebra (e.g., General Mathematics, Consumer Mathematics, Pre-Algebra, 
Algebra I) 

2. Pre-Calcidus Mathematics (e.g.. Geometry, Algebra II, Trigonometry, Pre-Calculus) 

3. College-Level Mathematics (e.g.. Calculus, Linear and Abstract Algebra, Probability and 
Statistics, Analysis, Discrete Mathematics) 

4. Computer Science 

5. Do not CTirrently teach 

6. Other (please specify) 



Thank you for completing this inventory. 

Please return it in the enclosed envelope or send it to: 

Educational Testing Service, 11*R 
Princeton, NJ 08541 
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Appendix D 

Cover Letters and Follow-Up Postcard 
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Cnver Letter for Primary Sample 



NATIONAL COUNCIL OF 



^aclie i^ of SVlathematic^ 






1906 Association Drive, Reston, Virginia 22091 ■ (703) 620-9840 



December 1989 



Dear Colleague: 

I am writing to ask your cooperation in a project that should be of utportance 
to members of the National Council of Teachers of Mathematics. 

Educational Testing Service (ETS) is in the process of developing a new 
generation of licensing tests for the National Teachers Examinations (NTE) . 

One of these new tests is the NTE Mathematics test. ETS has asked for our 
help in the design and content of this examination. 

As part of the developmental process, ETS has worked closely with teachers, 
college faculty, and school administrators to identify potentially uportant 
tasks. The enclosed questionnaire has been developed as a way to obtain your 
judgments on the inportance of these tasks for beginning teachers. The data 
obtained from these questionnaires will be used to define the content of the 
new examination. 

I urge you to please take the time to complete the enclosed questionnaire. A 
preliminary study has indicated that the survey takes approximately 30 minutes 
to complete. 

Please return your completed questionnaire within 10 days in the enclosed 
postage-paid envelope. 

Your cooperation and participation in this inportant project is appreciated. 

Sincerely, 



James D. Gates 
Executive Director 
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Cover Letter for Supplemental Sample 



KOLK^ATIONAl. TKSTING SKRVICE 




PHI NGF:T0N. N.,I. 08541 



C>0<f’<f21‘9000 
tO’9- 734 ‘1090 <Faxi 
CABLE-EDUCTESTSVC 

• 

DIVISION OF APPLIED 
MEASUREMENT RESEARCH 



Ma.y 1991 



Dear Colleague: 

I am writing to ask your cooperation in a project that should be of importance to 
teachers, college faculty, administrators, and other professionals in your field. Educational 
Testing Service (ETS) is in the process of developing a new generation of assessments for 
the purpose of licensing teachers. One type of assessment will be created to measure the 
prospective teacher’s subject-matter or specialty-area knowledge and will likely be 
administered upon completion of the undergraduate teacher education program. One 
such assessment is a new version of the NTE Mathematics test. I am asking for your help 
as we develop this examination. 

As part of the developmental process, ETS has worked closely with an advisory 
committee of classroom teachers, college faculty, and school administrators to identify 
potentially important knowledge and skill areas in mathematics instruction. The enclosed 
inventory has been constructed as a way to obtain your judgments on the importance of 
these areas for the newly licensed (certified) mathematics teacher. Your responses and 
those of other professionals to this inventory will guide the development of the new 
examination. 

You will notice that the inventory asks for some background information about you; 
this is solely for purposes of describing respondents. Your answers will be treated in strict 
confidence. 

A postage-paid envelope is enclosed for the return of your completed questionnaire. 
Thank you for your participation in this important project. 

Sincerely, 

Scott Wesley, Ph.D. 

Associate Research Scientist 



Enc. (2) 





Follow-Up Postcard for Primary and Supplemental Samples 



TASK ANALYSIS INVENTORY FOR 
MATHEMATICS TEACHERS 



Dear Colleague: 

I recently sent you an inventory to obtain your opinions of what a newly-liccnscd 
mathematics teacher should know and be able to do. If you have not already done 
so, please complete the inventory and return it in the postage*paid envelope to: 

Educational Testing Service 
Mail Stop 11-P 
Princeton, NJ 08541 

If you have already returned the inventory, please accept my thanks for your help 
in this important project. 

Sincerely, Scott Wesley, Ph.D. 

Associate Research Scientist 
Educational Testing Service 
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Appendix E 

Demographic Distributions 





Primary Sample 


Supplemental Sample 




(N » 462) 


(N « i 


J2) 


Number 


Percent 


Number 


Percent 


AGE (years) 










Under 25 


1 


0.2 


1 


1.2 


25-34 


2 


0.4 


25 


30.5 


35-44 


49 


10.6 


33 


40.2 


45-54 


202 


43.7 


17 


20.7 


55-64 


153 


33,1 


3 


3.7 


65 and over 


45 


9.7 


1 


1.2 


No response 


10 


2.2 


2 


2.4 


SEX 










Female 


160 


34.6 


43 


52.4 


Male 


291 


63.0 


37 


45.1 


No response 


11 


2.4 


2 


2.4 


RACE/ETHNICITY 










American Indian or Alaskan Native 


2 


0.4 


1 


1.2 


Black or African American 


13 


2.8 


5 


6.1 


Mexican American or Chicano 


2 


0.4 


2 


2.4 


Oriental or Aslan American 


6 


1.3 


1 


1.2 


Puerto Rican 


0 


0.0 


1 


1.2 


Other Hispanic or Latin American 


1 


0.2 


3 


3.7 


White 


419 


90.7 


64 


78.0 


Other 


3 


0.6 


3 


3.7 


No response 


16 


3.5 


2 


2.4 


HIGHEST EDUCATIONAL ATTAINMENT 










Less than Bachelor’s 


0 


0.0 


0 


0.0 


Bachelor’s 


0 


0.0 


7 


8.5 


Bachelor’s + Credits 


29 


6.3 


35 


42.7 


Master's 


27 


5.8 


9 


11.0 


Master’s + Credits 


257 


55.6 


29 


35.4 


Doctorate 


139 


30.1 


1 


1.2 


No response 


10 


2.2 


1 


1.2 
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Primary Sample 
(N - 462) 


Supplemental Sample 
(N » 82) 


Number 


Percent 


Number 


Percent 


CURRENT EMPLOYMENT STATUS 










Temporary Substitute 


1 


0.2 


0 


0.0 


Permanent Substitute 


0 


0.0 


1 


1.2 


Regular Teacher (not a substitute) 


180 


39.0 


76 


92.7 


Principal or Assistant Principal 


6 


1.3 


0 


0.0 


School Administrator 


2 


0.4 


0 


0.0 


Curricular Supervisor 


11 


2.4 


3 


3.7 


State Administrator 


9 


1.9 


0 


0.0 


College or University Faculty 


159 


34.4 


0 


0.0 


Other 


81 


17.5 


1 


1.2 


No response 


13 


2.8 


1 


1.2 


TEACHING EXPERIENCE 










Less than 1 year 


0 


0.0 


2 


2.4 


1-2 years 


1 


0.2 


13 


15.9 


3-5 years 


2 


0.4 


21 


25.6 


6-10 years 


14 


3.0 


14 


17.1 


11-15 years 


24 


5.2 


11 


13.4 


16-20 years 


39 


8.4 


10 


12.2 


21 or more years 


371 


80.3 


10 


12.2 


Never taught mathematics 


1 


0.2 


0 


0.0 


No response 


10 


2.2 


1 


1.2 


GRADES CURRENTLY TEACHING ’ 










Preschool/Wndergarten 


4 


0.9 


0 


0.0 


Grades 1 - 4 


5 


1.1 


0 


0.0 


Grades 5 - 8 


38 


8.2 


6 


7.3 


Grades 9-12 


192 


41.6 


78 


95.1 


College 


197 


42.6 


2 


2.4 


Do not currently teach 


53 


11.5 


1 


1.2 


Other 


32 


6.9 


3 


3.7 


No response 


0 


0.0 


1 


1.2 



^ Multiple responses were allowed. Total will not add up to 462 In the main sample and 82 In the supplementary sample. 
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Primary Sample 
(N « 462) 


Supplement 
(N » 


al Sample 
82) 


Number 


Percent 


Number 


Percent 


CURRENT TEACHING ASSIGNMENT ’ 










Arithmetic and Basic Algebra 


135 


29.2 


65 


79.3 


Pre-Calcuius Mathematics 


215 


46.5 


49 


59.8 


College*ljevel Mathematics 


184 


39.8 


10 


12.2 


Computer Science 


52 


11.3 


10 


12.2 


Do not currently teach 


59 


12.8 


1 


1.2 


Other 


78 


16.9 


7 


8.5 


No response 


0 


0.0 


1 


1.2 


GEOGRAPHIC REGION 










Northeast 


100 


21.6 


24 


29.3 


Central 


120 


26.0 


18 


22.0 


South 


118 


25.5 


23 


28.0 


Far West 


113 


24.5 


16 


19.5 


No response 


11 


2.4 


1 


1.2 




Multiple responses were allowed. Total will not add up to 462 in the main sample and 82 In the supplementary sample. 
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Appendix F 

Importance Ratings for Primary and Supplemental Samples 
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Teaching 

Sex Race/ Experience 

Ethnicity Geographic Region (teachers only) 

F M POC W NE C S FW 0-5 6+ 

Explain conditions under which a continuous function does not have a derivative 2.97 2.89 3.00 2.91 2.96 2.88 2.93 2.92 2.58 2.83 
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